AbiPy v0.9.1 @IMEC¶
M. Giantomassi and the AbiPy group¶
10th international ABINIT developer workshop
May 31 - June 4, 2021 - Smart Working, Lockdown@BE
- These slides have been generated using jupyter, nbconvert and revealjs
- The notebook can be downloaded from this github repo
- To install and configure the software, follow these installation instructions
Use the Space key to navigate through all slides and SHIFT + Space key to go back one slide.
What is AbiPy?¶
Python package for:¶
- Generating ABINIT input files automatically
- Post-processing results extracted from netcdf or text files
- Interfacing ABINIT with external tools such as vesta, wannier90, lobster, etc.
- Running ABINIT-specific workflows on laptops as well as on HPC clusters
Dependencies:¶
- Hard deps: pymatgen, netcdf4, matplotlib, plotly, numpy, scipy, pandas, ipython …
- Soft deps: ASE, phonopy, jupyter …
NB: AbiPy can be interfaced with other packages (e.g ASE, phonopy) via converters.
How to install AbiPy¶
Using conda and the conda forge channel (recommended):
conda install abipy --channel conda-forge
Since conda is not limited to python packages, one can install ABINIT in the same env with:
conda install abinit -c conda-forge
Using pip and python wheels:
pip install abipy --user
For further info see this installation howto
Jupyter notebooks with examples and lessons inspired by the official tutorials¶
In [4]:
%embed https://nbviewer.jupyter.org/github/abinit/abitutorials/blob/master/abitutorials/index.ipynb
Out[4]:
Since AbiPy is not restricted to high-throughput, we'll show how to use the terminal to analyze calculations
Well, a python script would be much more flexible but the goal here is to show that one can replace grep, vim, gnuplot with AbiPy
- Then we show how to use the python API for improved flexibility
No perl scripts were harmed in the making of this notebook
Command line interface¶
- abiopen.py ➝ Open output files inside ipython or print/visualize file
- abistruct.py ➝ Operate on crystalline structures read from file
- abicomp.py ➝ Compare multiple files (i.e. convergence studies)
- abiview.py ➝ Quick visualization of output files
- abinp.py ➝ Generate input files for typical calculations
- abigui.py ➝ Start GUI in web browser
Documentation¶
abistruct.py --helpfor manpageabistruct.py COMMAND --helpfor help aboutCOMMAND
HTML documentation available at http://abinit.github.io/abipy/scripts/index.html
Examples¶
abistruct.py spglib si_scf_GSR.nc
abistruct.py convert si_scf_GSR.nc -f cif
abiopen.py si_scf_GSR.nc --print
and many more...¶
abiopen.py
Print results stored in FILE (more than 45 file extensions supported)¶
In [5]:
!abiopen.py si_scf_GSR.nc --print
To produce a predefined set of matplotlib figures, use:¶
abiopen.py mgb2_kpath_FATBANDS.nc --expose --seaborn
Replace --expose with --notebook to generate a jupyter notebook with predefined python code¶
If you are still using perl scripts to extract data from the main output file, check this out:¶
abiopen.py gs_dfpt.abo --expose --seaborn
To visualize all the results inside the web browser, use¶
abiopen.py out_GRUNS.nc --expose-web # -ew for the short version
Creating dashboards from the command line¶
- Creating panels inside a notebook is great if you needs GUIs plus the possibility of executing python code.
- There are however cases in which we only need a dashboard with widgets to interact with the data.
Convert structure from netcdf format to CIF (abivars, xsf, poscar, qe, siesta, wannier90, json)¶
In [6]:
!abistruct.py convert si_scf_GSR.nc -f cif
Read structure from FILE and print ABINIT variables for NSCF along k-path:¶
In [7]:
!abistruct.py kpath si_scf_GSR.nc
Connect to the materials project database to find similar structures.¶
In [8]:
!abistruct.py mp_match si_scf_GSR.nc
Need to call anaddb to compute and visualize ph-bands and DOS from DDB?¶
abiview.py ddb ZnSe_hex_qpt_DDB --seaborn
Add --phononwebsite to visualize data on the phononwebsite by Henrique¶
In [9]:
!abiview.py --help
To compare multiple structures:¶
In [10]:
!abicomp.py structure *.cif si_nscf_GSR.nc *_DDB
AbiPy API¶
AbiPy plots with matplotlib¶
In [11]:
gsr = abiopen("si_nscf_GSR.nc")
gsr.ebands.plot(with_gaps=True);
AbiPy plots with plotly¶
In [12]:
gsr.ebands.plotly(with_gaps=True); # obj.plot becomes obj.plotly
To upload the plotly figure to the chart studio server, use:¶
gsr.ebands.plotly(with_gaps=True, chart_studio=True);
Users can finally customize the AbiPy plot without changing the python code 🎉¶
Interactive 3d plots with plotly:¶
In [13]:
gsr.ebands.kpoints.plotly(title="k-path in 3d with plotly");
and, ça va sans dire, phonons from DDB with plotly:¶
In [14]:
znse_ddb = abilab.abiopen("ZnSe_hex_qpt_DDB")
phbst_file, phdos_file = znse_ddb.anaget_phbst_and_phdos_files()
phbands, phdos = phbst_file.phbands, phdos_file.phdos
phbands.plotly_with_phdos(phdos, units="cm-1", title="ZnSe Phonon bands + DOS in cm-1");
In [15]:
paths = [
"mgb2_888k_0.01tsmear_DDB",
"mgb2_888k_0.04tsmear_DDB",
"mgb2_121212k_0.01tsmear_DDB",
"mgb2_121212k_0.04tsmear_DDB",
]
paths = [os.path.join(abidata.dirpath, "refs", "mgb2_phonons_nkpt_tsmear", f)
for f in paths]
robot = abilab.DdbRobot()
for i, path in enumerate(paths):
robot.add_file(path, path)
In [16]:
# Define function to change labels:
func = lambda ddb: "nkpt: %s, tsmear: %.2f" % (
ddb.header["nkpt"], ddb.header["tsmear"])
robot.remap_labels(func)
robot
Out[16]:
Now we can build a dataframe with the most important parameters:¶
In [17]:
robot.get_params_dataframe()
Out[17]:
and check that all DDBs have been computed with the same crystalline structure:¶
In [18]:
robot.get_lattice_dataframe()
Out[18]:
To analyze the effect of k-sampling/smearing on the vibrational properties:¶
In [19]:
# Invoke anaddb and store results
r = robot.anaget_phonon_plotters(nqsmall=2)
r.phbands_plotter.gridplot_with_hue("tsmear", with_dos=True);
Quick intro to AbiPy Flows and selected examples¶
⚠️ There are two different workflow infrastructures:¶
Internal AbiPy implementation (abipy.flowtk modules):¶
- ✅ Lightweigth, no database required
- ✅ Designed for rapid prototyping and/or for supporting advanced ABINIT capabilities
- ❌ No explicit support for high-throughput (HT) applications.
AbiFlows package (requires Fireworks and MongoDB database)¶
- ✅ HT-oriented: use MongoDB to store workflow status and results for further analysis
- ✅ High-level API designed for HT applications (e.g. phonon calculations for the materials project)
- ❌ Not all the flowtk workflows are immediately available in AbiFlows.
NB: There's an ongoing effort to reimplement AbiFlows in terms of the atomate framework. In this talk, we will mainly discuss the new features available in abipy.flowtk.
Effective masses with DFPT¶
This flow performs:¶
- GS-SCF run followed by NSCF run with k-path to locate band edges automatically
Compute $\epsilon^{\alpha\beta}_{n\bf{k}}$ and the effective mass tensor at the band edges using $|u_{n\mathbf{k}}\rangle$, and the k-derivatives $|u_{n\mathbf{k}}^\alpha\rangle$ $H^\alpha_{\mathbf{k}}$, $H^{\alpha\beta}_{\mathbf{k}}$
For the formalism, see J. Laflamme Janssen, et. al. Phys. Rev. B 93, 205147
- More that 1k systems already computed by J. C. Abreu.
Python API:¶
flow = flowtk.Flow("flow_effmass_dfpt")
# Build input for GS SCF calculation.
scf_input = make_scf_input()
# This object implements all the worflow logic
from abipy.flowtk.effmass_works import EffMassAutoDFPTWork
work = EffMassAutoDFPTWork.from_scf_input(scf_input)
flow.register_work(work)
Dependency Graph¶
- To run the flow with the high-level interface, use: run_effmass_dfpt.py --sheduler
- autoparal algorithm:
- select "optimal" Ncpus for each task
- set MPI-related variables in the input
- submit tasks and start monitoring
- indipendent tasks are executed in parallel
- Support for bluegene, moab, pbspro, sge, shell, slurm, torque
- Restart capabilities (e.g. timeout limit) and error handlers
To generate a GS-SCF input, one can use factory functions (HT-oriented)¶
In [21]:
from abipy.abio.factories import gs_input
gs_input(structure="si.cif", pseudos="14si.pspnc", ecut=8)
Out[21]:
or build it from scratch if full controll is wanted:¶
In [22]:
other_inp = abilab.AbinitInput(structure="si.cif", pseudos="14si.pspnc")
other_inp.set_vars(ecut=8, nsppol=2, toldfe=1e-8)
other_inp.set_autokmesh(8)
other_inp
Out[22]:
In [23]:
ddb = abilab.abiopen("ZnSe_hex_qpt_DDB")
#abilab.abipanel(); # Important
ddb.get_panel()
Out[23]:
Don't be surprised if you start to click buttons and nothing happens in the GUI!
One needs a python backend to execute the callbacks triggered by the widgets.
Documentation for these new features available at:¶
In [25]:
%embed https://abinit.github.io/abipy/graphical_interface.html
Out[25]:
How to run AbiPy Flows¶
In order to run a Flow, we need two configurations files:¶
scheduler.yml providing:
- scheduler parameters such as days, hours, minutes, max_njobs_inqueue, max_ncores_used, …
manager.yml providing:
- list of shell commands to be executed before running ABINIT
- list of modules to load
- options for the queue manager (bluegene, moab, pbspro, sge, shell, slurm, torque)
See this page for examples or use the abidoc.py script and the syntax:
- abidoc.py scheduler
- abidoc.py manager
- abidoc.py manager slurm
How to run calculations?¶
The simplest way to start the scheduler from the shell is via the syntax:
run_elastic.py --scheduler # -s for the short option
For non-trivial Flows, we suggest to put the scheduler in background and use nohup
so that we can disconnect from the shell session without killing the scheduler.
nohup run_elastic.py --s > log 2> err &
Obviously, it is possible to submit a Slurm script that executes the script on the compute note with 1 core.
To interact with the Flow, one can also use the abirun.py script, e.g:
abirun.py FLOWDIR status